Skip to content

[fix](be) Default to the Default chain when S3 role_arn is set without provider_type#60822

Open
CalvinKirs wants to merge 3 commits intoapache:masterfrom
CalvinKirs:master-be-aws-iam
Open

[fix](be) Default to the Default chain when S3 role_arn is set without provider_type#60822
CalvinKirs wants to merge 3 commits intoapache:masterfrom
CalvinKirs:master-be-aws-iam

Conversation

@CalvinKirs
Copy link
Member

@CalvinKirs CalvinKirs commented Feb 25, 2026 •
edited
Loading

closed #60501

One-line summary
On the BE side, setting AWS_ROLE_ARN no longer implicitly forces cred_provider_type to INSTANCE_PROFILE; if AWS_CREDENTIALS_PROVIDER_TYPE is not explicitly provided, it now defaults to the Default chain.

Behavior change

  • Before: AWS_ROLE_ARN only -> INSTANCE_PROFILE
  • After: AWS_ROLE_ARN only -> DEFAULT (Default chain)
  • Explicit config still wins: AWS_CREDENTIALS_PROVIDER_TYPE=WEB_IDENTITY -> WEB_IDENTITY

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@CalvinKirs
[fix](be) Default to the Default chain when S3 role_arn is set witho…
2ce3a3b 
…ut provider_type

  One-line summary
  On the BE side, setting AWS_ROLE_ARN no longer implicitly forces cred_provider_type to INSTANCE_PROFILE; if AWS_CREDENTIALS_PROVIDER_TYPE is not explicitly provided, it now defaults to the Default chain.

  Behavior change

  - Before: AWS_ROLE_ARN only -> INSTANCE_PROFILE
  - After: AWS_ROLE_ARN only -> DEFAULT (Default chain)
  - Explicit config still wins: AWS_CREDENTIALS_PROVIDER_TYPE=WEB_IDENTITY -> WEB_IDENTITY
@Thearas
Copy link
Contributor

Thearas commented Feb 25, 2026

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@CalvinKirs
Copy link
Member Author

CalvinKirs commented Feb 25, 2026

run buildall

@doris-robot
Copy link

doris-robot commented Feb 25, 2026

TPC-H: Total hot run time: 28615 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7daab178d8b0021219b868b17e63badd190495a1, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17627	4455	4270	4270
q2	q3	10643	760	531	531
q4	4682	395	258	258
q5	7573	1233	1005	1005
q6	174	179	149	149
q7	776	834	697	697
q8	9324	1451	1287	1287
q9	4808	4667	4674	4667
q10	6760	1865	1616	1616
q11	461	254	242	242
q12	706	563	468	468
q13	17767	4242	3416	3416
q14	228	222	212	212
q15	921	803	788	788
q16	718	714	673	673
q17	705	847	426	426
q18	6101	5388	5277	5277
q19	1264	978	609	609
q20	507	491	386	386
q21	4665	1832	1387	1387
q22	340	282	251	251
Total cold run time: 96750 ms
Total hot run time: 28615 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4416	4315	4334	4315
q2	q3	1776	2163	1724	1724
q4	868	1175	784	784
q5	4025	4286	4317	4286
q6	178	170	141	141
q7	1724	1591	1486	1486
q8	2401	2647	2510	2510
q9	7418	7446	7439	7439
q10	2607	2823	2438	2438
q11	527	446	416	416
q12	497	585	447	447
q13	3991	4439	3600	3600
q14	275	351	315	315
q15	849	788	797	788
q16	732	760	734	734
q17	1174	1633	1336	1336
q18	7072	6807	6566	6566
q19	874	945	969	945
q20	2100	2146	2011	2011
q21	3954	3536	3343	3343
q22	474	446	410	410
Total cold run time: 47932 ms
Total hot run time: 46034 ms

@doris-robot
Copy link

doris-robot commented Feb 25, 2026

TPC-DS: Total hot run time: 184396 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7daab178d8b0021219b868b17e63badd190495a1, data reload: false

query5	5240	637	516	516
query6	335	215	203	203
query7	4211	458	261	261
query8	343	254	238	238
query9	8764	2759	2745	2745
query10	563	364	329	329
query11	17032	16896	16603	16603
query12	189	124	122	122
query13	1257	433	332	332
query14	6800	3139	2987	2987
query14_1	2800	2792	2823	2792
query15	205	195	176	176
query16	989	469	462	462
query17	1064	742	611	611
query18	2713	443	342	342
query19	218	207	185	185
query20	139	127	129	127
query21	225	146	119	119
query22	5406	5618	5721	5618
query23	17612	17066	16971	16971
query23_1	17000	17048	17050	17048
query24	7559	1667	1312	1312
query24_1	1310	1303	1300	1300
query25	642	530	453	453
query26	1289	274	165	165
query27	2942	551	305	305
query28	4891	1910	1926	1910
query29	924	673	565	565
query30	321	275	228	228
query31	930	760	695	695
query32	81	74	69	69
query33	550	377	306	306
query34	1053	929	575	575
query35	618	658	583	583
query36	1074	1128	989	989
query37	132	95	84	84
query38	2942	2896	2883	2883
query39	883	884	830	830
query39_1	832	832	803	803
query40	223	149	133	133
query41	68	59	56	56
query42	108	105	104	104
query43	383	387	354	354
query44	
query45	197	187	179	179
query46	868	976	607	607
query47	2134	2107	2039	2039
query48	298	302	227	227
query49	618	456	393	393
query50	672	273	221	221
query51	4092	4046	4022	4022
query52	108	106	94	94
query53	289	334	301	301
query54	305	290	252	252
query55	89	78	78	78
query56	319	309	307	307
query57	1352	1327	1246	1246
query58	290	269	271	269
query59	2559	2680	2554	2554
query60	335	327	315	315
query61	145	142	146	142
query62	621	587	520	520
query63	308	276	275	275
query64	4789	1251	991	991
query65	
query66	1366	462	370	370
query67	16466	16421	16192	16192
query68	
query69	396	312	287	287
query70	980	994	966	966
query71	339	305	300	300
query72	2718	2656	2433	2433
query73	532	567	318	318
query74	9993	9883	9780	9780
query75	2832	2741	2436	2436
query76	2288	1032	679	679
query77	352	389	326	326
query78	11287	11493	10657	10657
query79	1245	774	599	599
query80	1370	625	511	511
query81	556	294	244	244
query82	978	147	121	121
query83	343	254	239	239
query84	253	120	95	95
query85	929	476	421	421
query86	431	312	290	290
query87	3126	3066	3026	3026
query88	3531	2652	2627	2627
query89	415	373	344	344
query90	1976	171	169	169
query91	165	160	138	138
query92	78	77	68	68
query93	995	836	496	496
query94	647	307	252	252
query95	586	394	319	319
query96	641	508	225	225
query97	2478	2451	2426	2426
query98	227	212	216	212
query99	1011	976	909	909
Total cold run time: 256276 ms
Total hot run time: 184396 ms

@hello-stephen
Copy link
Contributor

hello-stephen commented Feb 25, 2026

BE UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.57% (19589/37263)
Line Coverage 36.16% (182685/505154)
Region Coverage 32.50% (141755/436151)
Branch Coverage 33.46% (61455/183650)

@hello-stephen
Copy link
Contributor

hello-stephen commented Feb 25, 2026

BE Regression && UT Coverage Report

Increment line coverage 100% (0/0) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 57.38% (20941/36496)
Line Coverage 40.30% (202972/503654)
Region Coverage 36.98% (162847/440319)
Branch Coverage 37.65% (69362/184236)

@CalvinKirs
Copy link
Member Author

CalvinKirs commented Feb 26, 2026

run buildall

@doris-robot
Copy link

doris-robot commented Feb 26, 2026

TPC-H: Total hot run time: 28817 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6a6a09fec0d91666a1e3d9b356da6a741b568810, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17671	4403	4311	4311
q2	q3	10646	782	538	538
q4	4681	355	255	255
q5	7540	1216	1039	1039
q6	177	174	144	144
q7	779	848	685	685
q8	9819	1485	1327	1327
q9	5194	4753	4684	4684
q10	6845	1907	1634	1634
q11	486	256	251	251
q12	753	570	462	462
q13	17762	4236	3424	3424
q14	239	226	224	224
q15	990	804	798	798
q16	744	719	661	661
q17	713	881	415	415
q18	6101	5408	5330	5330
q19	1478	986	612	612
q20	509	488	394	394
q21	4566	1842	1386	1386
q22	343	281	243	243
Total cold run time: 98036 ms
Total hot run time: 28817 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4472	4320	4324	4320
q2	q3	1777	2167	1717	1717
q4	864	1158	754	754
q5	4037	4286	4338	4286
q6	186	174	141	141
q7	1723	1612	1479	1479
q8	2439	2677	2510	2510
q9	7569	7498	7435	7435
q10	2694	2844	2390	2390
q11	505	456	428	428
q12	514	584	449	449
q13	3932	4454	3701	3701
q14	309	317	281	281
q15	868	816	791	791
q16	746	775	721	721
q17	1197	1506	1277	1277
q18	7042	6826	6647	6647
q19	907	897	903	897
q20	2099	2209	1996	1996
q21	4118	3484	3299	3299
q22	437	448	389	389
Total cold run time: 48435 ms
Total hot run time: 45908 ms

@doris-robot
Copy link

doris-robot commented Feb 26, 2026

TPC-DS: Total hot run time: 184188 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6a6a09fec0d91666a1e3d9b356da6a741b568810, data reload: false

query5	4932	619	507	507
query6	327	252	211	211
query7	4211	464	266	266
query8	329	247	240	240
query9	8750	2754	2728	2728
query10	546	383	341	341
query11	17051	16845	16557	16557
query12	190	129	127	127
query13	1272	453	351	351
query14	6381	3204	2967	2967
query14_1	2825	2757	2826	2757
query15	208	195	179	179
query16	987	480	469	469
query17	1074	721	618	618
query18	2567	449	350	350
query19	214	210	184	184
query20	140	130	129	129
query21	225	151	123	123
query22	4944	5880	5416	5416
query23	17632	17218	17029	17029
query23_1	17075	17370	17208	17208
query24	7195	1609	1221	1221
query24_1	1234	1248	1233	1233
query25	581	482	428	428
query26	1233	273	156	156
query27	2765	519	281	281
query28	4425	1851	1866	1851
query29	781	562	463	463
query30	310	241	211	211
query31	871	726	649	649
query32	81	71	71	71
query33	515	335	282	282
query34	909	905	553	553
query35	637	663	599	599
query36	1087	1113	916	916
query37	144	97	79	79
query38	2948	2986	2872	2872
query39	890	856	855	855
query39_1	827	829	829	829
query40	227	154	135	135
query41	62	59	59	59
query42	115	105	103	103
query43	377	383	349	349
query44	
query45	195	191	183	183
query46	875	991	612	612
query47	2123	2169	2025	2025
query48	326	312	228	228
query49	624	469	393	393
query50	667	279	215	215
query51	4064	4047	4087	4047
query52	105	110	96	96
query53	291	335	283	283
query54	303	261	270	261
query55	92	85	82	82
query56	313	318	291	291
query57	1369	1308	1269	1269
query58	294	280	275	275
query59	2522	2631	2547	2547
query60	340	338	332	332
query61	149	148	157	148
query62	644	578	556	556
query63	313	282	278	278
query64	4868	1287	998	998
query65	
query66	1411	455	349	349
query67	16372	16371	16246	16246
query68	
query69	398	299	282	282
query70	952	1007	963	963
query71	337	312	292	292
query72	2830	2649	2442	2442
query73	535	538	318	318
query74	9993	9912	9789	9789
query75	2842	2735	2409	2409
query76	2314	1056	661	661
query77	355	406	317	317
query78	11240	11389	10742	10742
query79	1122	791	619	619
query80	1341	620	546	546
query81	545	280	249	249
query82	983	150	123	123
query83	339	260	240	240
query84	250	119	94	94
query85	913	491	452	452
query86	434	334	293	293
query87	3144	3106	2963	2963
query88	3497	2698	2627	2627
query89	427	366	353	353
query90	1987	182	176	176
query91	162	151	132	132
query92	87	71	71	71
query93	923	832	506	506
query94	646	319	298	298
query95	582	400	313	313
query96	637	510	235	235
query97	2494	2522	2406	2406
query98	237	213	233	213
query99	956	996	874	874
Total cold run time: 253259 ms
Total hot run time: 184188 ms

@hello-stephen
Copy link
Contributor

hello-stephen commented Feb 26, 2026

BE UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.54% (19576/37259)
Line Coverage 36.17% (182821/505431)
Region Coverage 32.49% (141835/436581)
Branch Coverage 33.49% (61583/183877)

@hello-stephen
Copy link
Contributor

hello-stephen commented Feb 26, 2026

BE Regression && UT Coverage Report

Increment line coverage 100% (0/0) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 63.35% (23120/36494)
Line Coverage 46.64% (235045/503940)
Region Coverage 43.75% (192813/440761)
Branch Coverage 44.92% (82856/184469)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

dev/3.1.x dev/4.0.x

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Bug](storage vault) S3 Storage Vault with role_arn broken in 4.0.3 - IRSA/WebIdentity credentials no longer work

4 participants

@CalvinKirs @Thearas @doris-robot @hello-stephen